35 results
9 - Choice: dynamics and decision rules
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 242-286
-
- Chapter
- Export citation
-
Summary
Learning in animals is studied mostly by behaviorists of one sort or another. Behaviorists differ among themselves, of course. Radical (Skinnerian) behaviorists look at things a little differently from associative learning (Hullian) behaviorists. And there are cognitive types who favor computer-like metaphors and presumptions about symbolic processing. Nevertheless, all agree in their disdain for the folk psychology of wishes, feelings, and conscious deliberation. No behaviorist would take as a model for choice Charles Darwin's agonizing about whether or not to marry. Darwin wrote copious notes, two months before his engagement to his cousin Emma Wedgwood, listing the pros and cons. A sample:
Marry [pros]:
Children – (if it Please God) [14] – Constant companion,
(& friend in old age) who will feel interested in one, –
object to be beloved & played with. – better than a
dog anyhow. – Home, & someone to take care of
house – Charms of music & female chit-chat. – These
things good for one's health. – [16] but terrible loss of time…
Not Marry [cons]: Freedom to go where one liked – choice of Society
& little of it. – Conversation of clever men at clubs –
Not forced to visit relatives, & to bend in every
trifle. – to have the expense & anxiety of children –
perhaps quarelling – Loss of time. – cannot read in
the Evenings – fatness & idleness…
This is choice indeed! No one would imagine that animals choose like this. But folk psychology often slips in unannounced. In his famous matching-law experiment, to make his procedure work, Richard Herrnstein added what is called a changeover delay (COD): neither response could be rewarded for a second or two after each change from one response to the other. The reason was to prevent a third response: switching. Matching is got only when switching is suppressed by the COD: “The precise correspondence between relative frequency of responding and relative frequency of reinforcement broke down when the COD was omitted.”
But a little thought reveals that the response of “switching” is entirely hypothetical. It was derived neither from direct observation nor from proven theory. It came from intuition. If you, like the pigeon, had to choose repeatedly between two options, you might well consider “switching” as a third option to be assessed for payoff along with the other two.
15 - Time and memory, II
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 421-444
-
- Chapter
- Export citation
12 - Stimulus control and performance
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 345-376
-
- Chapter
- Export citation
-
Summary
Animals develop an internal representation of their world that guides action. We do not know all the details, but it seems reasonable to assume that there is something fixed about any representation quite independently of the valences – values, goods and bads, motives – attached to aspects of it. Things look as they look, whether or not good or bad consequences are associated with them. The valences are essential, although they are often omitted from cognitive accounts. But without a motive, “the rat is left buried in thought at the choice point,” as Edwin Guthrie famously said of cognitive behaviorist E. C. Tolman.
Search images may be an exception, since the object before the viewer flips from being invisible to being seen. And the “flip” is often accompanied by reinforcement – the bird brushing a cryptic moth that flies off and is then “seen” and eaten. Representations of very complex objects may have to be acquired through a history of explicit reinforcement. Medieval teachers believed that Latin is learned only through the birch, and this general view of the motivation required for complex learning was almost universal until recently. Still, for recognition of simple stimuli, no special training seems to be necessary. The effect of reward and punishment is to give value to certain objects or places, as represented, rather than to create or modify the representations themselves.
Is performance then determined solely by the animal's representation? In this chapter, I argue that there is at least one other factor that must be taken into account: competition among activities for available time. These two factors, competition and external stimuli, taken together account for numerous experimental results on generalization and discrimination. The rest of the chapter explains how competition and stimulus control contribute to discrimination, behavioral contrast, generalization, and peak shift.
Inhibitory and excitatory control
Animals need to know both what to do and what not to do; hence, stimuli can have both inhibitory and excitatory effects. But, as we saw in earlier chapters, when an animal is not doing one thing, it is probably doing something else. Moreover, animals are highly “aroused” under the conditions typical of operant conditioning experiments – hunger, combined with frequent access to small amounts of food. Behavioral competition is then especially intense: The animals have a lot to do and limited time in which to do it.
13 - Molar laws
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 377-397
-
- Chapter
- Export citation
-
Summary
Every operant conditioning experiment is a feedback system. Response and reinforcer measures are neither independent nor dependent variables – causes and effects – but interdependent variables. Because of feedback, the relations between response and reinforcer rates, for example, reveal little about true causes, the processes internal to the organism that allow it to react to the stimuli and reinforcers the environment provides. But, also because of feedback, the relations between response and reinforcer measures in these experiments are often extremely orderly. This chapter describes a very simple model for the reliable molar regularities of simultaneous and successive free operant choice (concurrent and multiple variable-interval [VI] schedules). I seek the order in the empirical laws, skipping over the molecular/procedural details necessary to obtain them, such as the effects of changeover delay, interresponse time reinforcement, etc., which were discussed in Chapter 9.
Matching and optimality
Chapter 9 showed how diminishing marginal utility could account for response functions (steady-state response rate vs. reinforcer rate functions) obtained on numerous simple schedules. Chapter 10 showed how marginal utility can explain optimal choice between food patches. A version of this approach can be easily applied to concurrent schedules (simultaneous discrimination).
Matching can be derived in a way that allows for interim activities and can be generalized so as to account for performance on multiple schedules (i.e., successive discrimination). In a two-component concurrent schedule, the components share an interim activity, which occurs in both. I assume that in each component, the balance between operant reinforcement and the interim activity adjusts so as to minimize cost (maximize value), in the sense defined in Chapter 8.
The condition for optimal (maximizing) behavior is that the marginal change in reinforcement rate be equal to the marginal change in the value of the interim activity, formally
dR (x) /dx = dV (z) /dz,
where V(z) is the value of the interim activity as a function of its level, z, x is the level of responding in one component, and R(x) is its rate of reinforcement. If we assume that the marginal value of the interim activity is constant across both components, the optimality condition is that
dR (x) /dx = dR (y) /dy,
where y is the response rate in the other component.
Acknowledgments
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp xvi-xviii
-
- Chapter
- Export citation
4 - Direct orientation and feedback
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 67-101
-
- Chapter
- Export citation
-
Summary
Indirect orientation – kinesis – works by means of successive comparisons. Animals orient in this way either when the gradient to be detected is too shallow to permit simultaneous comparison or when the source of stimulation cannot be sensed at a distance because the direction of local gradients is not perfectly related to the direction of the source. Visual and auditory stimuli allow for a more efficient strategy: The local gradient always points to the source, and gradients are usually quite steep. Thus, simultaneous comparisons can provide immediate feedback about the proper direction of movement, allowing direct orientation with no wasted motion.
I next discuss simple mechanisms of direct orientation, first in a descriptive way, and then in terms of feedback mechanisms. The feedback discussion serves several purposes: It shows how feedback is involved in orientation, and how a mathematical model can account for experimental results. It shows how apparently different orientation reactions can be considered as variants of the same process. And it illustrates the difference between static and dynamic behavior theories. The remainder of the chapter discusses the type of explanation given by feedback accounts and shows how the concept of feedback control provides a mechanistic explanation for motive and purpose.
Taxes
Fraenkel and Gunn classify direct orientation (taxic) reactions into four main types. These do not, in fact, correspond to completely different mechanisms, but the mechanisms will be easier to understand after I describe the different types and the experiments used to tell one from the other.
Klinotaxis
The first taxic reaction, klinotaxis, is really an intermediate case, since it involves both direct orientation and successive comparisons like kineses. Figure 4.1 shows an example of the relevant behavior: Mast tested each of four maggots (Lucilia sericata) four to six times for its orientation to a horizontal beam of light. Movement was always directly away from the light, although the tracks are not perfectly smooth. The maggots, particularly individual A on the right, move their heads from side to side (wiggles in the record). These head movements provide a clue to the underlying process. Fraenkel and Gunn write: “During steady forward crawling the head is sometimes put down symmetrically in line with the axis of the body, but from time to time it comes down alternately to the right and to the left.
5 - Operant behavior
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 102-134
-
- Chapter
- Export citation
-
Summary
Some behavior makes sense in terms of the events that precede it; other behavior makes more sense in terms of the events that follow it. Reflexes are behavior of the first kind. The type and vigor of the reflex response are closely related to the type and intensity of the eliciting stimulus. Kineses are behavior of the second kind. The movement of the orienting bacterium from moment to moment is unrelated to any single stimulus, yet its behavior as a whole can be understood in relation to the prevailing chemical gradient: The organisms aggregate in high concentrations of an attractant and disperse away from concentrations of a noxious substance.
Behavior like this is guided by its consequences: Under normal conditions, the location of the bacterium is determined by the chemical gradients in its environment. Because the antecedent (proximal) causes of its behavior are many but the final cause is one, we can most simply describe the behavior in terms of its outcome: The bug finds the food.
Operant behavior and B. F. Skinner
Behavior guided by its consequences was called operant behavior by B. F. Skinner, and the term has stuck. The word operant refers to an essential property of goal-directed behavior: that it has an effect on the environment. If the bacterium cannot move, or if movement has no effect on the organism's chemical environment, then its behavior will not appear to be guided by a goal.
Skinner in his various writings has used the term operant behavior in three senses: (a) by exclusion, as behavior that is not tied to an eliciting stimulus. It is, if not spontaneous, at least behavior for which “…no correlated stimulus can be detected upon occasions when it is observed to occur.” (b) As behavior involving units, operants, defined by the correlation between stimulus and response. Skinner argued that both stimulus and response are classes of events, and the definition of each class is an empirical matter: Each should be defined in terms of the other so that the elements show “orderly changes” as a function of relevant variables. A stimulus becomes “that which causes a response,” and a response “that which is caused by a stimulus.” This kind of bootstrap definition may seem peculiar but is, in fact, quite characteristic of scientific theory generally.
Preface to the second edition
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp xiii-xv
-
- Chapter
- Export citation
-
Summary
The first edition of Adaptive Behavior and Learning was published at a time when research on animal learning was at its apogee. By 1983 the divisions of the field and its basic principles had already been established. Advances have been made since then in mapping neurophysiological underpinnings and in the elaboration and refinement of theoretical models. Many new experimental papers have been published. But relatively little has been added by way of fundamental principles or new behavioral techniques. Even without revision, therefore, AB&L had acquired relatively little of a Rip van Winkle aspect even after thirty years. This impression was confirmed as I continued to receive requests for the book even though it had long been out of print. Cambridge University Press issued a digitally printed edition of the original book in 2009. Hints that the book still had signs of life led me over the years to revise and modify it, leading to the creation of an internet edition in 2003, revised again in 2010.
In 2013, I approached Cambridge with the idea that they might be willing to convert my 2010 PDF file into a Kindle format, so the book could reach a larger audience. They declined, but after some discussion made a counter offer of a new paper edition. I happily agreed. But this new development forced me to reconsider the whole project.
I decided to do a complete revision. The original book had an odd format: relatively elementary chapters, each followed by a more advanced appendix. Citations were named in the text, in the standard style. I have modified both these features in this edition. The research literature on animal learning has grown vast. To cite every author in the text would be otiose – and distracting, encouraging the reader to focus on the names rather than the ideas. So citations are now largely confined to footnotes. The internet, which did not exist at the time of the first edition, now provides ready access to scholarly sources. It makes unnecessary a totally comprehensive reference list. As for the mathematical appendices, only Chapter 4 retains an appendix. Otherwise, I have incorporated that material, plus several new topics, into the chapters, some of which are slightly more mathematical than before.
19 - Learning, II
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 501-531
-
- Chapter
- Export citation
-
Summary
The study of classical/Pavlovian conditioning began in the reflex tradition. Physiologist Pavlov was interested in the brain, and the conditioned salivation of dogs was his way of studying it. But, when released from restraint, his dogs showed much other activity in anticipation of the food powder about to be dropped into their mouths. A visitor to Pavlov's laboratory reports the reaction when a dog, trained to salivate at the sound of a metronome, was released: The animal at once approached the metronome, wagged its tail, barked, and behaved in every way as it might toward an individual who could feed it. But gastric expert Pavlov regarded this behavior as a distraction. His dogs were therefore restrained in a special harness so that their saliva could be collected more easily. Yet salivation is the least important thing they learned. But it is, of course, the thing most easily observed, measured, and understood in reflex terms. And salivation is directed by the autonomic, involuntary nervous system, not the voluntary, somatic one.
B. F. Skinner, father of operant conditioning, was a great admirer of Pavlov and emulated him in many respects. He followed Pavlov's approach to conditioning in one way but not another. First, unlike both Pavlov and independent discover of the operant method D. C. Grindley, he did not restrain his animals. This freedom allowed Skinner to discover schedules of reinforcement and led to a rich evolution of the operant method. But Skinner accepted Pavlov's belief that classical conditioning is a separate system, dealing not with operants (voluntary responses – although Skinner avoided that word) but respondents, involuntary responses like salivation, the flick of the nictitating membrane, and the conditioned emotional response (CER).
The behavior of the dog released from Pavlov's harness shows that it had learned a great deal more than salivation. It is this learning, which is to do with behavioral variation – the origins of operant behavior rather than its selection by reinforcement – that is the hugely important effect of Pavlovian conditioning.
Did the dog released from his harness also salivate to the sound of the metronome? Probably not; probably the dog did not salivate until it actually reached the instrument (or the food bowl). Several elegant experiments have established that autonomic responses such as salivation tend to occur at the last predictable moment before food delivery.
14 - Time and memory, I
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 398-420
-
- Chapter
- Export citation
-
Summary
Memory is the most protean term in psychology. There are many kinds, apparently: long- and short-term memory; explicit and implicit memory; working and reference memory; episodic, semantic, event and procedural memory; primary and secondary memory. The plethora of terms suggests that the subject is not well understood. There is still no consensus on what we mean by memory, or on its relation to learning. I will try to make a few explicit connections in this chapter.
Chapter 5 defined memory/learning simply as a change of state caused by a stimulus: Memory is involved if how the animal behaves at time t2 depends on whether event A or event B occurred at previous time t1. Breaking a leg is a change of state in this sense, of course; it certainly will change future behavior. So we need to restrict the definition to effects that are specific and to some extent reversible: The difference in behavior at t2 should bear some sensible, informational relation to the difference between prior events A and B; and we should be able to change the effect by additional experience. Habituation, dishabituation, spontaneous recovery (“reminiscence”), and, particularly, control of behavior by elapsed time are all related to memory in this sense.
Much is known about timing in conditioning experiments. This chapter reviews the properties of temporal control and derives some general principles about the discrimination of recency. The next chapter shows that these principles also apply to more traditional situations used to study memory in animals, such as successive discrimination reversal, delayed matching to sample, and the radial-arm maze.
Temporal control
A PBS Nova TV program aired in 2014 introduced viewers to Jazz, a Hungarian Vizsla dog, belonging to a Scottish family of regular habits. Jazz seems to tell time. Every day at about 4:30 pm, he leaps up on to the sofa and looks out for Johnny, who always comes home at 5:00. It is as if he has a clock that tells him the time to expect the boss. How does he do it? Perhaps it was Johnny's wife, Christine, who comes home at 4:00, an hour earlier? Did Jazz just takes his cue from that? Or does Jazz really have some kind of internal clock?
17 - Learning, I
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 457-485
-
- Chapter
- Export citation
-
Summary
Animal learning linked to reward and punishment has been a major concern of psychologists since Thorndike and Pavlov. Unfortunately, research has tended to focus on ever-more standardized experimental “paradigms” in which one or two well-defined responses could be studied under highly controlled conditions. Behavioral variation, an essential feature of learning, has been largely painted out of the picture.
The field has become increasingly divided into two topics: classical (Pavlovian) conditioning and operant conditioning. Classical conditioning research is dominated by the idea of association, hence the label associative learning – although associative processes are presumed to operate also in operant conditioning. In fact, the division conforms to no characteristic of the processes involved. The distinction between classical and instrumental conditioning is a matter not of process but of procedure (open-loop vs. closed-loop, in the terminology of Chapter 6; see also Chapter 5). I have said much about operant conditioning in earlier chapters. Now it is time to look a bit more closely at classical conditioning and the differences between the two.
The division between classical and operant conditioners has led to different theoretical and practical preoccupations. As I pointed out in Chapter 9, operant conditioners have almost abandoned work on the process of learning and focus almost exclusively on steady-state adaptation to behavior that is demonstrably reversible – which at least allows them to work with individual animals rather than groups. The underlying state is certainly not reversible, as I also pointed out earlier. Classical conditioners retain an interest in a fundamentally irreversible learning processes, which has forced them to study and compare groups even though this method also has its problems.
Learning means change, and change has given psychologists the same kind of trouble that motion gave Zeno: How can one study motion, when a body must be in one place or another? How can something move at all? The Procrustean solution is to act as if there is a single learning “process” and then design our experiments so as to preclude any other. Some of the more tightly controlled conditioning procedures have this flavor.
A safer tack may be to look at a range of learning phenomena in different species and see what useful generalizations emerge. In this chapter, I look at the learning of bees, taste-aversion learning, and several experiments on classical conditioning.
Index
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 579-599
-
- Chapter
- Export citation
1 - The evolution, development, and modification of behavior
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 1-23
-
- Chapter
- Export citation
-
Summary
Organisms are machines designed by their evolution to play a certain role. The role, and the stage – the environment – where it is played, is called the organism's niche. For example, most cats – tigers, leopards, mountain lions – play the role of solitary hunters. Wolves and wild dogs are social hunters; antelope are social grazers; and so on. The basis for the modern idea of niche is Charles Darwin's discussion of an organism's “place in the economy of nature.”
A niche defines the pattern of behavior – the adaptive repertoire – compatible with an organism's survival and reproduction. A niche doesn't tell an organism how to behave. It just punishes it – by death or reproductive failure – for doing the wrong thing. A niche is a filter not a creator. Niches are best defined by example. It is pretty obvious that the talents required of a good leopard are quite different from those needed by an effective antelope. For leopards, powerful means of attack, a digestive system attuned to meat, and a visual system adapted to attend to one thing at a time work well. But a prey animal like an antelope needs a good way to evade attack, a lengthy gut able to cope with poor herbivore diet, and a visual system able to detect threat from any quarter. Hence, the claws and teeth of the leopard, its forward-facing eyes and short digestive tract, as well as the rapid running and maneuvering of the antelope, its lengthy gut, and sideways-facing eyes – all have an obvious functional explanation.
The behavioral adaptations required by different niches are usually less obvious than morphological (form) differences, especially if they involve the ways that past experience affects present potential, that is, differences in learning. The match between adaptation and niche is no less close because it is hard to see, however.
For simple niches, such as those filled by most nonsocial invertebrates, a set of built-in responses to commonly encountered environments suffices to get the organism to its evolutionary goal, which, for bugs as much as billionaires, is survival and reproduction (Darwinian fitness). The animal need only avoid bad things and approach good ones, all signified by signals innately coded. Stimulus–response mechanisms, plus some sensitivity to rates of change, are sufficient for a wide range of surprisingly intelligent behavior.
Frontmatter
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp i-iv
-
- Chapter
- Export citation
2 - Variation and selection: kineses
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 24-37
-
- Chapter
- Export citation
-
Summary
The simpler the animal, the more we know about how it works: what makes it behave at all and what selects one behavior over others. This chapter illustrates the processes of variation and selection of individual behavior in some very simple cases – orientation mechanisms in plants and protists. The next chapter describes elementary processes such as habituation, adaptation, and the “laws” of reflex action, which are some of the ingredients of adaptive behavior in higher animals. Chapter 4 introduces the notion of feedback in connection with direct (taxic) orientation.
Simple orienting mechanisms
Finding the proper habitat, a place not too hot or cold, nor too dry or wet, safe from predators, and with a supply of food, is a major behavioral problem for all organisms. For simple, single-celled animals such as protists and primitive invertebrates, it is the main problem. Since these organisms possess either no or only the most rudimentary nervous system, they must get nourishment and stay safe using very simple means. Their simple orientation mechanisms exhibit the properties of adaptive behavior in its clearest form.
The movement of climbing plants provides a good illustration of the orientation problem, and of a simple solution to it. An important objective for any green plant is to gain maximum access to sunlight. Obstacles are presented by other plants with similar requirements. Where the need for light is not outweighed by other considerations, such as avoidance of predation or extremes of temperature or the effects of wind, plants therefore grow vertically and seek the highest, best-lighted point.
Darwin, an experimenter as well as a theorist, identified rotation of the growing plant tip (he called it circumnutation) as a key element in directed plant growth. He describes circumnutation in a hop plant:
When the shoot of a hop (Humulus lupulus) rises from the ground, the two or three first-formed, whilst very young, may be seen to bend to one side and to travel slowly round towards all points of the compass…From seven observations made during August…the average rate during hot weather and during the day is 2 hrs. 8 m. for each revolution…The revolving movement continues as long as the plant continues to grow; but each separate internode, as it becomes old, ceases to move.
7 - Feeding regulation: a model motivational system
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 175-204
-
- Chapter
- Export citation
-
Summary
The operant behavior of higher animals depends on their motivation, the value of the reward or punishment. If we ignore for the moment the problem of competition among motivational systems (hunger vs. sex vs. thirst, etc.), there are two aspects to motivation: “How hungry am I?” and “How hard should I work to get food?” The answer to the first question depends entirely on the animal's internal state; it is a regulatory question. But the answer to the “How hard should I work to get food?” question depends both on the animal's state and on the opportunities offered by the environment: It is worth working hard for food if much food is to be got. It is not worth working hard if the probability of payoff is low (on the other hand, if you are hungry and more work means more food, you may work hard even for small amounts). This aspect of motivation is known as incentive.
This chapter deals with regulation and incentive. I take feeding as a typical motivational system and show how taste, body weight, and even drugs and brain lesions combine to affect how much rats will eat and how hard they will work for food. The aim is to dissect a single motivational system and show how a few relatively simple principles underlie diverse effects. The next chapter deals with interactions among motivational systems.
Reinforcement and homeostasis
Animals must eat to live and, on an operant food schedule, must work to eat. It would be astonishing, therefore, if schedule performance had nothing to do with the regulation of nutritional balance. Nevertheless, the study of reinforcement schedules has been pursued largely in isolation from regulatory considerations. For example, open-economy experiments, where the animal is maintained at a reduced weight throughout brief daily experimental sessions, usually find a positive relation between the rate of an operant response and the rate or amount of food delivery. On the other hand, in closed economies, where the animal lives in the experimental apparatus and gets all its food from the reinforcement schedule, rate of operant responding is inversely related, or unrelated, to rate of food delivery. This difference should not surprise. In an open economy, the animal cannot regulate his food intake (the experimenter does it for him); in a closed economy, he must do it himself.
20 - Learning, III: procedures
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 532-562
-
- Chapter
- Export citation
-
Summary
This chapter analyzes a number of widely used experimental procedures from the point of the principles derived in previous chapters.
Conditioned reinforcement
Conditioned reinforcement is invoked when a response produces a stimulus that signals reinforcement. The signal stimulus is termed a conditioned or secondary reinforcer. How useful is this idea?
There is no doubt that the conditioned reinforcement procedure can act as an aid to memory. I look at that property first. But is a conditioned reinforcer really a reinforcer? Or is there a better way to look at the data? It turns out that relative proximity to the (primary) reinforcer explains most schedule effects usually attributed to conditioned reinforcement.
Acquisition shows what I mean by a memory effect. For example, suppose we attempt to train a pigeon to peck a white key for food reward. The food is available for a peck every 30 sec (say). Food is not delivered immediately, but only after a delay of 5 sec. Even if the animal does occasionally peck the key, the effect of each reward is likely to be small, the bird will obtain little food, and pecks may be few. The likely reason is that it is difficult for the bird to pick out the peck, a brief event preceded and followed by other activities, as the best predictor of food, unless peck and food occur close together in time.
We can make the bird's task much easier by arranging that pecks on the white key are immediately followed by a 5-sec green stimulus ending in (response-independent) food. The time relations between pecking and food are exactly the same as in the original delayed-reward procedure, yet the pigeon will rapidly learn to peck the white key to get the 5-sec of green as a signal. (Since this is an autoshaping procedure, the bird will also peck the green key, but those pecks are not necessary for the effect.)
The two-stimulus procedure is termed a one-link chained schedule: peck → stimulus → food. The schedule in the white stimulus is fixed interval (FI) with production of the green stimulus as the reward; the schedule in green is 5-sec fixed time (FT). The green stimulus is used as a reinforcer and seems to act like one. However, it gains its power not innately but by virtue of its pairing with food. It is thus a conditioned or secondary reinforcer.
18 - Models of classical conditioning
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 486-500
-
- Chapter
- Export citation
-
Summary
Classical conditioning was first studied extensively by I. P. Pavlov, who placed it firmly in the reflex tradition codified and summarized by C. S. Sherrington (Chapter 3). These two men set the study of conditioning on a trajectory that eventually led to the split between Pavlov's classical and Skinner's operant conditioning. Chapter 19 deals with this split. In this chapter, I will describe reflex-type theoretical models for conditioning – what might be called the classical conditioning models. One of these, the Rescorla–Wagner (R–W) model, has been hugely influential. The number and complexity of proposed conditioning models has increased greatly since the original R–W chapter in 1972. I will try just to extract the essentials.
Inference and classical conditioning
But first, some comments on inference. The occurrence of a hedonic event (i.e., an unconditioned response [US]) such as food or shock poses a Bayesian problem for the animal: Which environmental feature, of those present now and in the past, is the most likely cause of the US? Is there a conditioned stimulus (CS), and if so, what is it?
The animal has two kinds of information: prior knowledge, and temporal and frequency relations between the valued event (US) and its potential cause (an environmental stimulus, the CS – or an operant response). When, and how often, did the potential CS occur in relation to the US? This chapter deals mostly with ‘how often.’
Prior knowledge is of two kinds: innate priors and previously formed representations. The animal's evolutionary history provides it with predispositions to connect certain kinds of event with certain kinds of outcome: Pain is more likely to be associated with an organism than with an inanimate stimulus – recall the hedgehog experiment. A related finding is shock-induced fighting: A pair of rats placed in a cage and briefly shocked through their feet will usually attack one another. Pain, in the presence of a probable cause, elicits the behavior that is preprogrammed for that situation: Attack the cause. Both the quality and the intensity of the US will determine the likely cause.
If the situation is familiar, or is similar (in terms of the animal's representation – see Chapter 11 for a discussion of similarity) to a familiar one, the probable cause can be inferred.
Contents
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp v-xii
-
- Chapter
- Export citation
21 - Comparative cognition
- J. E. R. Staddon, Duke University, North Carolina
-
- Book:
- Adaptive Behavior and Learning
- Published online:
- 05 March 2016
- Print publication:
- 07 March 2016, pp 563-578
-
- Chapter
- Export citation
-
Summary
In recent years, public interest has been aroused by the exploits of smart animals – surprisingly intelligent behavior by apes, dogs, fish, and even parrots, one of whom [which?] earned an obituary in a distinguished news weekly. Much of this interest is scientific: developmental and evolutionary. Can the precursors of uniquely human behavior like language and higher cognition be discerned in animals, especially animals closely related to us? Not that close relationship is essential. Perhaps there has been convergent evolution? Octopus eyes resemble human eyes (theirs are superior in at least one respect, actually), but similar selection pressures have molded both. Aspects of bird-song development resemble some features of human language learning, as we saw in Chapter 16. But birds have only the remotest evolutionary relationship to humans. Dogs have evolved as human companions and have become uniquely sensitive to human expressions and gestures (not to mention odors!). Evolution is involved, but not common descent. Perhaps comparison with animals can help us understand the evolutionary fundamentals that lie behind all intelligence?
Another impulse that underlies research on comparative cognition is more lofty, however. The aim is to understand the animal mind, and see how close it is to the human mind. I have elsewhere labeled some of this work (perhaps unfairly) animal behavior as entertainment – or perhaps as “Dr. Dolittle research” – experiments that explore the extent to which animals can do things that previously seemed unique to people – like talk, count, form concepts, be embarrassed, feel guilty, deceive others, have a ‘theory of mind,’ or do ‘mental time travel.’ Whales give out mysterious calls that can travel many miles underwater; they stand on their heads and slap their huge tails in purposeful ways. What are they saying? Vervet monkeys emit three kinds of alarm call, depending on the type of predator they are warning about. But are they really intending to inform? Or is it just an automatic house-burglar-alarm–type system? Crows drop walnuts on a busy highway – so that they can be cracked by passing cars? Maybe. Some very ingenious experiments have been done to get at questions like this. Hard-line behaviorists find some of the work frustrating, but whatever the criticism, it surely has value as natural history, exploring the limits of what animals can be trained to do.